Viewer-adjusted stereoscopic image display

ABSTRACT

A stereoscopic video playback device is provided that processes original stereoscopic image pairs taken using parallel-axis cameras and provided for viewing under original viewing conditions by scaling and cropping to provide new viewing condition stereoscopic video on a single screen.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present patent application is a continuation application of U.S.patent application Ser. No. 16/783,784 filed Feb. 6, 2020, now allowed,which, in turn, is a continuation application of International PatentApplication no. PCT/CA2018/051039 filed on Aug. 29, 2018, designatingthe United States and claiming priority of U.S. provisional patentapplication No. 62/551,942 filed on Aug. 30, 2017 and U.S. provisionalpatent application No. 62/627,825 filed on Feb. 8, 2018, the contents ofwhich are hereby incorporated by reference in their entirety.

TECHNICAL FIELD

This application relates to stereoscopic image displaying.

BACKGROUND

Stereoscopic video or cinematography is an art. The arrangement of thecameras to obtain left and right video streams for playback with thebest 3D effect is not a trivial task. The arrangement of the camerasrequires knowledge of the cinema and viewing audience arrangement, aswell as an understanding of how 3D is perceived by most people undersuch viewing conditions. It is generally accepted in the art ofstereoscopic cinematography that two cameras are arranged with theiroptical axes to be convergent, as this will result in the best 3Deffect. The camera separation and convergence angle is selected withknowledge of the average viewer distance from the display screen,average eye separation and average viewing position with respect to thecenter of the screen. If these viewing conditions are not respected,then the quality of the 3D experience is compromised.

When the viewing conditions are to be changed from the ones originallyintended by the stereoscopic cinematographer, it is known in the art toreformat the stereoscopic video for the new viewing conditions.Reformatting typically involves analyzing the stereoscopic image pairsto determine the depth of individual pixels, and then generatingstereoscopic image pairs using the original image and the depthinformation so as to be able to recreate a suitable stereoscopic imagestream for the new viewing conditions. Such reformatting iscomputationally extensive and is performed for the new viewingconditions. When the viewing conditions change, the computationallyextensive process is repeated.

SUMMARY

Applicant has discovered that any potential loss of quality in the 3Dexperience caused by using parallel, non-convergent cameras is overcomeby the increase in quality of the 3D experience when such stereoscopicvideo is reformatted for the viewing conditions of the viewer on asingle screen.

Accordingly, a playback device is provided that processes originalstereoscopic image pairs provided for viewing under original viewingconditions by scaling and cropping to provide new viewing conditionstereoscopic video on a single screen.

In order to avoid reformatting of the stereoscopic images as describedabove, it is possible to display stereoscopic images intended originallyfor display with a first field of view on a new single display having asecond field of view.

Applicant has further discovered that acquiring and storing 3D imagesusing parallel axis cameras with a wider field of view that is normallyexpected to be used for viewing is advantageous to be able to processthe 3D images recorded at the viewing device (or within the viewingsystem) for viewing under a greater range of viewing conditions.

A broad aspect is a method of processing stereoscopic images for displayto a viewer on a single screen, the stereoscopic images taken usingparallel-axis cameras having a first field of view. The method includesusing a definition of a second field of view provided by the singlescreen, an interocular distance Io for the viewer and a distance betweenthe viewer and the single screen to position and to scale thestereoscopic images so that display of the images on the single screenat the distance from the viewer respects the first field of view, andwhen the stereoscopic images as scaled for the screen are larger thanthe screen, to crop the images for the screen, and when the stereoscopicimages as scaled for the screen are smaller than the screen, providing aborder for the images for the screen.

In some embodiments, the method may include selecting a zoom windowwithin the stereoscopic images to thus change the first field of view,wherein the stereoscopic images may be scaled respecting the changedfirst field of view.

In some embodiments, the zoom window may be offset from a center of thestereoscopic images to permit viewing a region of interest within thestereoscopic images.

In some embodiments, the viewer input may be used to move the offsetwhile viewing the stereoscopic images.

In some embodiments, the stereoscopic images may be still images.

In some embodiments, the stereoscopic images may be video images.

In some embodiments, the stereoscopic images may be converted tocombined anaglyphic format images.

In some embodiments, the stereoscopic images may be converted to columninterleaved format images for display on an autostereoscopic display.

In some embodiments, the stereoscopic images may be converted to asequence of page-flip images for viewing with shutter glasses.

In some embodiments, the stereoscopic images may be converted to asequence of line-interleaved for polarized displays.

In some embodiments, the method may include acquiring user input toobtain the definition of a second field of view provided by the singlescreen.

In some embodiments, the method may include acquiring sensor data toobtain the definition of a second field of view provided by the singlescreen.

In some embodiments, the stereoscopic images may be positioned on thesingle screen to correspond to an object separation of Io between righteye and left eye images for distant objects.

In some embodiments, the viewer may include a plurality of viewers, andthe interocular distance Io may be selected to be a smallest interoculardistance among the plurality of viewers.

In some embodiments, the stereoscopic images may be further scaledand/or positioned using a relative base offset to make the most distantobjects appear closer to the screen and/or to make the closest objectsappear closer to the screen. The objective is to reduce possible eyestrain due to a difference in ocular accommodation for focussing on thesingle screen and ocular accommodation for focussing on close and/or farobjects. In this further scaling and positioning of the relative baseoffset, it is possible to maintain objects appearing at a depth of thesingle screen to appear at a same depth.

Another broad aspect is a device for processing stereoscopic images fordisplay to a viewer on a single screen, the device comprising aprocessor and a memory readable by the processor, the memory storinginstructions for performing the method as defined herein.

Another broad aspect is a computer program product comprising anon-transitory memory storing instructions for a processor orreconfigurable hardware for performing the method as defined herein.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be better understood by way of the following detaileddescription of embodiments of the invention with reference to theappended drawings, in which:

FIG. 1A is a diagram of an exemplary parallel camera system;

FIG. 1B is an illustration of a left camera image above a right cameraimage;

FIG. 1C is an illustration of how qualitatively each image is placedwithin a single screen's frame with an appropriate sideways offset tocorrespond the viewer's interocular distance in which the magnificationis one and the display field of view is greater than the capture fieldof view;

FIG. 1D illustrates schematically changes in field of view with viewingdistance to a screen;

FIG. 1E illustrates schematically changes in field of view for a fixedviewing distance with changing screen size;

FIG. 1F is an illustration of how qualitatively each image is placedwithin a single screen's frame with an appropriate sideways offset tocorrespond the viewer's interocular distance in which the magnificationis one and the display field of view is smaller than the capture fieldof view;

FIG. 1G is an illustration of how qualitatively each image is placedwithin a single screen's frame with an appropriate sideways offset tocorrespond the viewer's interocular distance in which the magnificationis 1.5 and the display field of view is about the same as the capturefield of view;

FIG. 1H is an illustration of how qualitatively each image is placedwithin a single screen's frame with an appropriate sideways offset tocorrespond the viewer's interocular distance in which the magnificationis 0.5 and the display field of view is about the same as the capturefield of view;

FIG. 2 is a diagram illustrating proportions tied to the calculation ofthe parallax of an exemplary parallel camera system;

FIG. 3A is a diagram of dual parallel screens S1 and S2 placed before auser;

FIG. 3B is a diagram showing proportions for calculating the perceiveddistance of an object Op of dual parallel screens S1 and S2 placedbefore a user;

FIG. 4A is a diagram corresponding to width perception in the realworld;

FIG. 4B is a diagram corresponding to width perception with monoscopicvision in the real world and the perceived world;

FIG. 5 is a diagram of a left eye's screen that is one of the dualscreens of a stereoscopic system where a line is occupying exactly theright half of the image displayed on the screen;

FIG. 6A is a diagram showing proportions of an object perceived in thereal world at a distance Drn;

FIG. 6B is a diagram showing how an object is perceived on the leftscreen of a dual screen system in the perceived world;

FIG. 7 is a diagram of dual screens S1 and S2 of an exemplarystereoscopic system, where S1 and S2 are perpendicular to the imaginaryline Io between the right and left eye, and S1 and S2 are centered onthe pupil of the left eye and the right eye respectively;

FIG. 8A is a diagram of dual parallel screens S1 and S2 placed before auser;

FIG. 8B is a diagram of different proportions relating to where anobject Op will be perceived when a user is facing a dual screen systemat a distance Ds from the eyes of viewer;

FIG. 9A is a diagram of two theoretical overlapping screens S1′ and S2′situated further away from the user than the dual screens S1 and S2;

FIG. 9B is a diagram showing proportions tied to how an object Op willbe perceived by the right eye on at least portions of the dual screensS1′ and S2′;

FIG. 9C is a schematic image from a left eye camera including a distantsun near the optical axis and midfield tree along the optical axis;

FIG. 9D is a schematic image from a right eye camera having a parallelaxis to the optical axis of the left eye camera, thus showing a distantsun in vertical alignment with the optical axis and the midfield treeoffset to the left;

FIG. 10A is a diagram of a single screen with two overlapping sectionsS1′ and S2′ situated further away from the user than dual screens S1 andS2;

FIG. 10B is a diagram showing different measurements tied to how anobject Op will be perceived by the user on the single screen with twooverlapping sections S1′ and S2′;

FIG. 10C is a schematic left eye image for display on a common screen,the image corresponding to the camera image of FIG. 9C with theinterocular offset Io/2 to the left included in the display image, theimage showing the distant sun in vertical alignment with the opticalaxis and the midfield tree in vertical alignment with the optical axisof the left eye;

FIG. 10D is a schematic right eye image for display on a common screen,the image corresponding to the camera image of FIG. 9D with theinterocular offset Io/2 to the right included in the display image, theimage showing the distant sun in vertical alignment with the opticalaxis and the midfield tree offset to the left of the optical axis of theright eye, the interocular distance Io being shown between the distantsun objects in the images of FIGS. 10C and 10D;

FIG. 11A is a diagram illustrating a simple screen system where portionsof the screens S1′ and S2′ are shared by both eyes;

FIG. 11B is a diagram illustrating measurements of a simple screensystem corresponding to portions of the screens S1′ and S2′ that areshared by both eyes;

FIG. 12A is a diagram illustrating a partial image of the image intendedfor the left eye by using a simple screen having the same ratio Ds/Ls1′as the dual screen system;

FIG. 12B is a diagram illustrating a partial image of the image intendedfor the right eye by using a simple screen having the same ratio Ds/Ls1′as the dual screen system;

FIG. 13 is a diagram illustrating the section of the original image thatwill be viewed as the left eye final image and the right eye finalimage;

FIG. 14A is a diagram of an exemplary single screen system, where thewidth of the screen S1′ is compared to the effective width of the image,where the image perceived by the left eye does not need to be adjustedbecause Lse=Ls1′;

FIG. 14B is a diagram of an exemplary single screen system, where thewidth of the screen S1′ is compared to the effective width of the image,where the image perceived by the left eye requires black strips to beadded on both sides because Lse<Ls1′;

FIG. 14C is a diagram of an exemplary single screen system, where thewidth of the screen S1′ is compared to the effective width of the image,where the image perceived by the left eye needs to be cut becauseLse>Ls1′;

FIG. 15A is a diagram of an exemplary single screen system perceived bya user where Lse=Ls1′;

FIG. 15B is a diagram of an exemplary single screen system perceived bya user where Lse<Ls1′;

FIG. 15C is a diagram of an exemplary single screen system perceived bya user where Lse>Ls1′;

FIGS. 16A and 16B are block diagrams of an exemplary stereoscopic systemfor cropping and scaling an image to be viewed on a display; FIG. 16Cillustrates the image source is a large field of view source, that isable to provide a desired image for a given viewing direction bycropping or de-warping and cropping;

FIG. 17A is a diagram of image acquisition or rendering using a virtualcamera in an exemplary volume reader method;

FIG. 17B is a diagram of single screen formatting in an exemplary volumereader method;

FIG. 18A is a graph of object depth when the depth of scale is 1, namelywhere objects are perceived at the same distance as in the real world;

FIG. 18B is a graph of object depth when the depth of scale is less than1, namely where objects are perceived closer than in the real world;

FIG. 18C is a graph of object depth when the depth of scale greater than1, namely where objects are perceived farther than in the real world;

FIG. 19A illustrates the geometry of Io and Bo for the images displayedon the screen;

FIG. 19B illustrates the resulting change in object width as a result ofmodifying Bo;

FIG. 20A illustrates the geometry of a distant object when Bo is equalto Io and

FIG. 20B illustrates the impact when using an optical base Bo lower thanIo on the appearance of the distant object;

FIG. 21A is a graph showing perceived space not being linear;

FIG. 21B is a graph illustrating the ratio between distances in the realworld versus the perceived world;

FIG. 22 is an illustration of a close object partially out of the fieldof view that causes difficulty in perceiving the depth of the closeobject because of the conflict with the associated screen edge or frame;

FIG. 23A illustrates schematically the vergence distance and the focaldistance that are the same for real world viewing, and FIG. 23Billustrates schematically the vergence distance and the closer focaldistance of the screen in stereoscopic 3D viewing;

Variables (upper case) Annotations (lower case) CAPTURE F = Focal s =screen G = Magnification se = effective image on B = Base or inter-axialthe screen c = camera/sensor SCREEN S = Screen p = user perception M =Magnification img = image Io = Interocular distance g = left C = Centerof the screen d = right COMMON P = Parallax h = horizontal D = Distancev = vertical L = Width R = Resolution CALCULATIONS Esp = spatial scaleEpr = depth scale

FIG. 24A is a drawing illustrating the viewing geometry for a case ofconvergence on behind of the screen,

FIG. 24B is a drawing illustrating the geometry of the screenconvergence angle and of the object convergence angle;

FIG. 25A illustrates the viewing geometry for a case of convergence infront of the screen; and

FIG. 25B illustrates the geometry of the screen convergence angle and ofthe object convergence angle.

FIG. 26 illustrates an exemplary embodiment of a system for streamingstereoscopic images in which a remote client display communicates with aserver to obtain the formatted stereoscopic images based on its displayparameters.

DETAILED DESCRIPTION

Prior to describing the geometry behind the image processing techniquesinvolved in embodiments described herein, a qualitative overview of theimage processing is presented.

In FIG. 1A, there is shown schematically how parallel cameras, namely aleft camera and a right camera can be arranged to capture the samescene. The two cameras can have the same properties of resolution,focus, field of view, etc., and they have parallel optical axes. The twocameras can be separated by a distance that can correspond to theseparation between a viewer's eyes. Objects at infinity appear in eachcamera image at the same position. Closer objects will have a differentparallax depending on the inter-camera distance, the position of theobject within the field of view including the distance of the objectfrom the camera.

In FIG. 1B, there is shown a left eye image above a right eye image withthe sun appearing at the same position in each image. A tree appearingin midfield will have a different position in the two images due to theparallax.

As shown in FIG. 1C, modification of the camera images of FIG. 1B isrequired for display on a single screen. Viewing on a single screen canbe done by known techniques. For example, anaglyphic color filterglasses can be worn by the viewer and the screen image is then composedof both right and left image data that is color encoded. In page-flipoperation, the viewer can wear shutter glasses that allow the right andleft eyes to see in alternating time slots while the screen imagealternates between right and left images. In an autostereoscopicdisplay, the viewer does not need any glasses, but the screen includes alens or screen mask that allows the right eye to see right eye pixelsand the left eye to see left eye pixels.

The field of view (FOV) of the display screen in FIG. 1C is larger thanthe original FOV of the camera images. As illustrated in FIG. 1C,qualitatively each image is placed within the single screen's window orframe with an appropriate sideways offset to correspond the viewer'sinterocular distance. This distance can vary from person to person. Asdescribed below, when a screen is to be viewed by two viewers, it may bebest to use the smallest interocular distance of the viewers to avoiddiscomfort for that viewer. The images thus obtained are displayed onthe single screen according to the stereoscopic display technique.

It will be appreciated that the FOV of the display or screen changes asillustrated in FIG. 1D as a viewer changes his or her distance from thescreen. When the viewer is closer to the screen, the FOV is greater.Likewise, when a viewer is at a fixed distance from a screen, the FOV isgreater with a larger screen than for a smaller screen. FIGS. 1D and 1Eare important for understanding qualitatively the dependence betweenFOV, the viewing distance and the screen size.

In the embodiment of FIG. 1C, the display screen provided an FOV greaterthan the original FOV, and thus some padding or blacking out of a borderportion can be done. In the embodiment of FIG. 1F, the display FOV issmaller than the capture FOV. This means that the display screen isessentially too small for the viewing distance. As illustratedqualitatively in FIG. 1F, cropping of the original capture images isdone so that the two images can be combined and fit onto the displayscreen. While some edge portions of the original capture images arelost, the images are stereoscopically faithful to the original capture.

In the embodiment of FIG. 1G, the stereoscopic output is to be magnifiedby a factor of 1.5. Qualitatively, one can see that the images of FIG.1B (repeated on the drawing sheet for ease of understanding) are firstenlarged and from the enlarged images, a portion able to fit the displayscreen is extracted and placed with the suitable interocular offset (Io)in the single display screen according to the stereoscopic displaytechnique. While the display screen FOV can be the same as the captureFOV, as a result of the magnification, a significant border portion ofthe capture images are lost. However, the stereoscopic effect of themagnified images is pleasant to view.

When the images are scaled in the way shown in FIG. 1G, themagnification affects the size of the objects and the perceivedparallax, thus making objects seem closer, and gives the impression thatthe zoom involved getting closer to the objects in the images. While theperception of the depth variation among the objects in the scene isreduced or flattened, because the images remain aligned with the twoeyes, the 3D effect works well in spite of the magnification.

The ability for the stereoscopic effect to withstand the adjustment tothe original capture images in accordance with the variable viewingconditions is facilitated by the original capture images being fromparallel axis cameras. It will be appreciated that cameras having axesthat are close to parallel provides images that are suitable.

It will be appreciated that the magnification of the capture imageillustrated in FIG. 1G need not be taken about the center of the captureimages, and that a window of interest is effectively selected whenperforming such a magnification. It will further be appreciated thatthis feature allows a viewer to move the window of interest in a waythat simulates panning of the original capture scene.

In the embodiment of FIG. 1H, the stereoscopic output is to be magnified(namely shrunk) by a factor of 0.5. Qualitatively, one can see that theimages of FIG. 1B (repeated on the drawing sheet for ease ofunderstanding) are first scaled-down and the smaller images are placedwith the suitable interocular offset (Io) in the single display screenaccording to the stereoscopic display technique. While the displayscreen FOV can be the same as the capture FOV, as a result of thereduction, no portion of the capture images is lost. The magnificationfactor can be chosen so that the images fit exactly the available FOV ofthe single display screen. As before, while the perception of the depthvariation among the objects in the scene is increased in the embodimentof FIG. 1H, because the images remain aligned with the two eyes, the 3Deffect works well in spite of the magnification.

Having described certain embodiments qualitatively, other embodimentswill be described below using exact geometry calculations.

Capture with Parallel Cameras

A stereoscopic capture system consisting of two identical cameras isarranged in a parallel configuration. As shown in FIG. 1A, the parallaxof an object captured with such a stereoscopic system is the differencemeasured between the positioning of this object on the image picked upby the left camera and the positioning of this same object on the imagecaptured by the camera on the right.

As shown in FIG. 2, it is defined that these two cameras have sensors ofwidth Lc and a focal length of length F. The centers of their respectivelenses are placed at a distance B from each other. This distance iscalled the base.

When an object is exactly on the center axis of the left camera, thenthis object will be represented exactly at the center of the sensor onthe left camera. The parallax of this object will be equal to thedistance between the point formed by this object on the sensor of thecamera on the right and the center of the sensor of the camera on theright, which is illustrated in the graph above by the segment pc. Theparallax of an object situated at a distance Dn can be determined bycomparing the equivalent right triangles where the sides Pc and B arecorresponding, as well as the sides Drn and F. Pc=B*F/Drn is obtained.

Depth Scale—Parallel Screens

To view stereoscopic images, parallel screen systems can be used, whereboth eyes display a separate image (left and right images) on their ownscreen. These two screens of identical size (named S1 and S2) arealigned directly with the center of the pupil of each eye (see FIG. 3A).

Using an object whose representation on the left screen would be locatedin Og, directly on the axis perpendicular to the screen and passing byits center, the representation on the right screen would be located inOd or at a Distance Ps (screen parallax) from the center of the screen.The perceived distance of the object Op given by the information ofdisparity would thus be Dp. There are two equivalent right triangles,and by matching the sides, the following ratios are obtained:

$\frac{{Io} - {Ps}}{{Dp} - {Ds}} = \frac{Io}{Dp}$

The following simplification may be made:

Dp(Io − Ps) = Io(Dp − Ds) Dp.Io − Dp.Ps = Dp.Io − Ds.Io Dp.Ps = Ds.Io$\begin{matrix}{{Dp} = \frac{{Ds}.{Io}}{Ps}}\end{matrix}$

The parallax on the screen (Ps) can be calculated by multiplying theparallax of the sensor (Pc) by a magnification factor on the screen.This magnification factor (M) corresponds to the ratio of the effectivewidth of the image presented on the screen (Lse) to the width of thecaptured image, which for all practical purposes is equal to the widthof the sensor (Lc). In the present case, let us establish that the imagepresented on the screen is the image originally captured in itsentirety. The following is obtained:Ps=Pc*MPs=Pc*(Lse/Lc)Ps=(B*F/Drn)*(Lse/Lc)Ps=(B*F*Lse)*(Drn*Lc)

The following is obtained by combining the two preceding equations:

Dp = Ds * Io/Ps Dp = Ds * Io/(B * F * Lse)^(*)(Drn * Lc)${Dp} = {\frac{Io}{B}*\frac{Ds}{Lse}*\frac{Lc}{F}*{Drn}}$

For a given spectator seated at a fixed distance from a givenstereoscopic screen and looking at content shot with a fixed base, itcan be said that Io, B, Ds, Lse, Lc and F are constant. The equationthen comes down to:Dp=C*DrnorDp/Drn=C=Éprand

${\overset{\prime}{E}{pr}} = {\frac{Io}{B}*\frac{Ds}{Lse}*\frac{Lc}{F}}$

In other words, the depth perception represented by the stereoscopicsystem is linearly proportional to that of the real world and its depthscale is equal to C. For any unitary variation of distance in the realworld (capture), there will be a variation of C of the distanceperceived by the observer (visualization). If:

-   -   Épr=1, then the perception of depth will be identical to that in        the real world;    -   Épr<1, then the observer will perceive a world shallower than        reality;    -   Épr>1, then the observer will perceive a world deeper than        reality.

Spatial Scale—Parallel Screens

In order to establish the real width of an object observed in the realworld, it is essential to know the relative distance. Indeed, inmonoscopic vision, objects of different sizes placed at differentdistances could give the impression of being of the same size. This isillustrated in FIGS. 4A and 4B, where the lines Lr1, Lr2 and Lr3 are allthree of different lengths but appear being of identical length for theobserver who does not know the relative distances and is thus inmonoscopic vision, Lp1=Lp2=Lp3.

The perceived width of an object is therefore directly related to thedistance information with respect to the observer of this object.

As shown in FIG. 5, a stereoscopic image is taken that is displaying aline occupying exactly the right half of the image displayed on thescreen of the left eye of a stereoscopic system with parallel screens.The width of this line on the screen is thus equal to Lse/2.

In stereoscopy, this line may be at an apparent distance different fromthe distance separating the observer from the screen. As shown in FIGS.6A and 6B, it can be assumed that this line is perceived at a distanceDp from the observer. At this perceived distance, the line will have aperceived width of Lp. In other words, it will be perceived that thisline is much wider because it is located much further.

As there are two equivalent right triangles, it may be established:

$\frac{Lp}{Dp} = \frac{{Lse}/2}{Ds}$ ou${Lp} = \frac{{Lse}*{Dp}}{2*{Ds}}$

It was shown above how to calculate the perceived depth (Dp) of anobject in such a stereoscopic system, and by substituting thiscalculation for the term Dp in the equation above, the following isobtained:

${Lp} = {\frac{Lse}{2*{Ds}}*\frac{lo}{B}*\frac{Ds}{Lse}*\frac{Lc}{F}*{Drn}}$${Lp} = {\frac{Io}{B}*\frac{Lc}{F}*\frac{Drn}{2}}$

Now, the width of the line in the real world (Lrn) can be determinedthat formed the line on the image of the left eye of perceived width Lp.As this line completely occupies the right part of the image of the lefteye, it can be established that it occupies entirely half of the sensorof the camera having captured this image as presented in the graph onthe previous page. It can be established, by applying Thales' theorem,that:

$\frac{Lrn}{Drn} = \frac{{Lc}/2}{F}$ or ${Lrn} = \frac{{Lc}*{Drn}}{2*F}$

It can be established that the scale ratio between the perceived widthof this line and any object in the perceived world (Lp) and their realworld equivalent (Ln) is as follows:

$\frac{Lp}{Lrn} = {\frac{Io}{B}*\frac{Lc}{F}*\frac{Drn}{2}*\frac{2*F}{{Lc}*{Drn}}}$$\frac{Lp}{Ln} = \frac{Io}{B}$${\overset{\prime}{E}{pr}} = \frac{Io}{B}$

In other words, the width perception represented by the stereoscopicsystem is linearly proportional to that of the real world and itsspatial scale is equal to Io/B. For each variation of width in the realworld (capture), there will be a variation of Io/B of the widthperceived by the observer (visualization). If:

-   -   Esp=1, then the perception of width will be identical to that in        the real world;    -   Esp<1, then the observer will perceive a world narrower than        reality (i.e. squeezed);    -   Esp>1, then the observer will perceive a wider world than        reality (expanded).

Proportionality of the Stereoscopic Representations

By knowing the depth and spatial scales of stereoscopic representations,a scale of proportionality of this representation can be established.This scale of proportionality aims to determine whether therepresentation will be flattened, lengthened or proportional to reality.The proportion ratio is established as follows:Z=Épr/ÉspIf:

-   -   Z=1, then the observer will perceive a proportioned world        (desired);    -   Z<1, then the observer will perceive a flattened world (more        comfortable, less effect);    -   Z>1, then the observer will perceive a stretched world (more        expansive, more spectacular)

For Z to equal 1, Épr=Ésp and therefore:

${\frac{Io}{B}*\frac{Ds}{Lse}*\frac{Lc}{F}} = \frac{Io}{B}$ THEREFORE${{\frac{Ds}{Lse} = \frac{F}{Lc}}\mspace{20mu}{AND}\mspace{14mu}{Global}\mspace{14mu}{scale}} = {{Io}/B}$

In other words, the captured field given by the Focal and Sensor pairare equal to the field of view of the stereoscopic system given by theimage width (screen) and image distance pair. Any system with screensparallel to the Ds/Lse proportions, regardless of their size, willprovide an equally equivalent experience, the proportion of which willbe given by Io/B (from ant to giant). For example, a stereoscopic imagecaptured for screens with a width of 10 m with an observer placed at adistance of 30 m from the screen (Ds/Lse=3) will give an identicalstereoscopic experience on 10 cm screens placed at 30 cm from the screen(Ds/Lse=3).

However, there is a problem tied to the fact that beyond a certain size(wider than Io), the parallel screens will touch each other. Unlessusing magnifying lenses, such as stereoscopes or virtual realityheadsets, the use of parallel displays is impractical, which greatlylimits their use. The next section explains how to bypass thislimitation and use the parallel cameras method for representations onmuch larger single screens such as 3DTVs or movie screens.

Conversion for Single Screens

The equations developed above work only for parallel screens, that is tosay for screens perpendicular to the imaginary line separating the twoeyes and whose centers are located exactly in the center of the pupil ofeach eye.

It has been demonstrated above that stereoscopic representations withsystems with the same Ds/Lse ratio (the ratio of the distance to thescreen over the width of the image presented on the screen) wouldprovide an experience in all regards identical, that is to say that theperceived size and distance of the objects would be perfectly identical.

As shown in FIGS. 8A and 8B, an Op object whose perceived distance is Dpis taken. This point would be represented on the screen of the left eye(S1) by the point Og and in the screen of the right eye (S2) by thepoint Od. The point Og is located exactly on the central axis of thescreen S1 while the point Od is situated at a distance Ps from thecenter of the screen S2. The two eyes thus converge at the point Opwhich will be the place where the observer will perceive that this pointis localized as illustrated in FIG. 8B.

As shown in FIGS. 9A and 9B, two theoretical screens S1′ and S2′ whichhave the same ratio Ds/Lse as the screens S1 and S2 and which aresituated farther away from the screens are taken. These screens aretheoretical since they overlap, which is not possible in the real world.It is therefore known that Ls1/Ds1 is equal to Ls2/D2 which are alsoequal to Ls1′/Ds1′ and Ls2′/Ds2′. Since the screens S1 and S1′ arecentered on the pupil of the left eye, it can be asserted that thepoints Og and Og′ will both be located on the central axis of theirrespective screens S1 and S1′. The points Od and Od′ will berespectively located at a distance Ps and Ps' from the center of thescreen S2 and S2′ as illustrated in FIGS. 9A and 9B.

For the point Op to be perceived at the same place in the tworepresentations, the points Od and Od′ form the same angle or that theratio Ps/Ds2 are equal to Ps'/Ds2′. It Is known that as S2′ is a linearmagnification of S2, Ps' will undergo the same magnification incomparison with Ps. In other words, Ls2′/Ls2=Ps'/Ps. It is also knownthat Ls2′/Ds2′=Ls2/Ds2 since the system was designed on the basis ofthis constraint. So it can be deduced that:Ls2′/Ds2′=Ls2/Ds2ThusLs2′/Ls2=Ds2′/Ds2SoLs2′/Ls2=Ps'/PsAndPs2′/Ds2′=Ps2/Ds2

The Op object will therefore be perceived in the same place when usingeither of these two systems. It has been therefore demonstrated that thetwo systems will offer an identical and equivalent stereoscopicexperience in all regards.

As shown in FIGS. 9C and 9D, the images captured using parallel axiscameras have distant objects, like that of the sun, at the sameposition, while closer objects are in different positions. Such imagescan be viewed using a head-mounted display.

It will be appreciated from FIG. 9B that the image seen from each eyecan be sized or scaled to fit a screen placed at a first depthcorresponding to where objects Od and Og are found or to fit a screenplaced at a second depth corresponding to where objects Od′ and Og′ arefound. Thus, a small screen used at the first depth can be replaced by alarger screen at the second depth that provides the same field of view.The scaling of the image for the larger screen may cause Od to change toOd′, however, the stereoscopic position of object Op does not change.

As will be understood from the description with reference to FIGS. 10 to13 below, the images of FIGS. 9C and 9D can be scaled as a function ofscreen position as described above, however, the scaling of images 10Cand 10D adversely affects the interocular distance, and thus any scalingalso requires a position offset (or maintaining the position of theleft-eye axis and right-eye axis during the scaling process) to maintainthe interocular distance.

If a different screen size is desired at either of the two depths,scaling of the images changes the field of view. With monocular viewing,viewing is generally more appreciated when a normal field of view isprovided, and the resolution is of good quality. Nonetheless, a viewercan sit closer or farther away from a screen, or change a 30″ screen fora 50″ screen at the same viewing distance, and the ability to see themonocular image is not adversely affected by changing the field of viewof the original image. With stereoscopic viewing, changing the field ofview will degrade the perception of stereoscopic depth of the objects.

For example, with reference to FIG. 9B, if the image presented to theright eye were presented on a larger screen at the second depth, Od′would appear further to the left as a result of the scaling to fit thelarger screen at the same second depth. Because the object Og′ willremain at the same central position within the left eye image, the depthof object Op will thus appear closer. This would create a distortion ofthe stereoscopic viewing.

If indeed a larger screen is to be used at the second depth, the largerscreen can be used to display the same field of view image withoutadversely affecting the stereoscopy. This can involve providing a borderaround the images on the screen. The effective field of view is notchanged.

If a smaller screen is to be used at the second depth, the smallerscreen can be used to display a portion of the image. This is likelooking at the world through a smaller window in the sense that theobjects seen on the smaller display are the same size as the objectsseen on the larger display, while only a portion of the original fieldof view is seen on the smaller screen. When the smaller screen has thesame resolution as a larger screen, the image will be magnified andcropped so as to maintain the same object sizes and to display theportion of the image. The edges of the original images will be lost,however, the stereoscopic effect will not be distorted due to the use ofthe smaller screen at the second depth.

Now that this equivalence has been established, it can now be transposedto a stereoscopic system based on a single screen as shown in FIGS. 11Aand 11B. To do this, a single screen such as two partial sections of twoseparate screens and centered on each pupil as shown in FIGS. 11A and11B is considered. Indeed, a screen S is taken whose center is locatedon the axis perpendicular to the two eyes of an observer and in thecenter of them. It can be said that this screen is the partialrepresentation of the screen S1′ (right part) as well as of the screenS2′ (left-hand part) and that the field of vision for each eye isasymmetrical (wider-on one side than the other from the center of eacheye).

As shown in FIGS. 10C and 10D, when images are to be seen dichopticallyon a same display, for example using anaglyphic glasses to view ananaglyphic image (e.g. cyan for the right eye and red for the left eye),LC shutter glasses to view alternatingly presented left-eye andright-eye images, or an autostereoscopic display, the images containdistant objects with a disparity of Io. The images captured usingparallel axis cameras have distant objects, like that of the sun, at thesame position once the offset Io is taken into account, while closerobjects are in respectively different positions with respect to thedistance objects.

It will be appreciated that the scaling of the stereoscopic images takenwith a camera for a first field of view for display on a screen for aviewer having a second field of view is not limited to displaying theentirety of the stereoscopic images. As shown in FIGS. 9C, 9D, 10C and10D, a region of interest zoom window can be selected within the sourcestereoscopic images. This window provides a smaller first field of viewthan the whole source image, however, the window can then be taken asthe source image and displayed as set out herein. The result ofselecting a window can be that there is less cropping of the images tofit the new screen.

This window selection need not be at the center of the images, and isillustrated to be somewhat to the left of the images in the Figures.This window selection can be controlled by user input to allow fornavigation of the direction of looking at the window within the sourceimages.

Thus, the source images can be wide angle or panoramic scenes, and thewindow selection can allow the viewer to explore the scenes by changingthe direction of viewing within the scenes.

As illustrated in FIGS. 12A and 12B, to obtain an experience equivalentto that of a parallel screen system by using a single screen having thesame ratio Ds/Ls1′, a partial image of the image intended for the lefteye (right part) and a partial image of the image for the right eye(left part) are presented, the partial images calculated in thefollowing way.

For the left eye, the right half of the width of the screen S1′ is equalto the width of the screen divided by two (Ls/2) plus the interoculardistance divided by two (Io/2) as shown in FIGS. 12A and 12B. Thecomplete screen width S1′ is thus equal to Ls/2+Io/2 multiplied by two,which gives Ls1′=Ls+Io. Since only the right-hand part of the image ofthe left eye can be displayed on the screen (Ls1′−Io), a part of theleft image equivalent to Io is cut. The image is cut according to theproportion Io/Ls1′ or Io/(Ls+Io).

Consider, for example, a 1920×1080 resolution image that should bepresented on a 140 cm wide screen, with the observer having aninterocular distance of 6.5 cm. The left portion of the image for theleft eye should be cut by 85 pixels:Io/(Ls+Io)*Rimg_h6.5 cm/(140 cm+6.5 cm)*1920 pixels=85.1877 pixels

To maintain the aspect ratio of the original image, the image is cut inthe vertical axis by the same proportion either:Io/(Ls+Io)*Rimg_v6.5 cm/(140 cm+6.5 cm)*1080 pixels=47.9181 pixels

As shown in FIG. 13, the final image for the left eye will therefore bea section of the original 1835×1032 image. It should be noted that thevertical portion of the image can be any part of the image (top, bottom,center, etc.) as long as you respect the number of pixels and take thesame selection for both eyes (stereoscopic alignment). To obtain theimage of the right eye, simply take the equivalent left section of theoriginal image of the right eye either a section of 1835×1032 with thesame vertical alignment as the section of the eye left.

These images can then be brought back to the resolution of the screen onwhich they will be displayed without having any impact on the finalstereoscopic result. The formulas for obtaining the final horizontal andvertical resolutions of the image are thus:Rimg_h′=Rimg_h*(1−Io/(Io+Ls))Rimg_v′=Rimg_v*(1−Io/(Io+Ls))

This method therefore makes it possible to use capture systems withparallel cameras for display with simple screens such as 3D televisionsor 3D cinema screens with a user experience that is equivalent in everyrespect.

Adaptations for Non-Identical Ds/L and F/Lc Ratios

It has been established above that it is possible to obtain astereoscopic experience proportional to reality (Z=1) when the ratioDs/Lse is identical to the ratio F/Lc. However, there may be constraintsin the stereoscopic display system that may make it impossible to meetthis ratio. It is nevertheless possible to modify the images so as torecover this ratio and the desired stereoscopic proportionality. Forpurposes of simplicity, a parallel screen system is used. A virtualreality headpiece is provided to a user with a very wide field of viewthanks to magnifying lenses. The final screen width is given by theformula Ls1′=Ls1*G, where G represents the magnification provided by thelens used.

Step 1: Determining the Effective Width

The effective width of the stereoscopic image is determined byconsidering the distance from the observer to the screen. For thispurpose, the following formula is used:F/Lc=Ds/LseSoLse=Ds*Lc/F

Step 2: Comparing:

The width of the screen S1′ is then be compared with the effective widthof the image. As shown in FIGS. 14A, 14B and 14C, if:

-   -   Lse=Ls1′, then the image can be displayed as it is on the        screen;    -   Lse<Ls1′, then reduce the size of the image (black bars,        centered window);    -   Lse>Ls1′, then you have to cut the image to respect the actual        size of the screen.

Step 3(A): Adjusting the Image when Lse<Ls1′

Method 1:

In this case, black bands can be added all around the image to keep theimage centered on the eye and to retain the original aspect ratio of theimage. To do this, the following formulas are used:Rimg_h′=Ls1′/Lse*Rimg_hRimg_h′−Rimg_h=Rimg_h*(Ls1′/Lse−1)(Rimg_h′−Rimg_h)/2=Rimg_h/2*(Ls1′/Lse−1): horizontal black bandsANDRimg_v′=Ls1′/Lse*Rimg_v(Rimg_v′−Rimg_v)/2=Rimg_v/2*(Ls1′/Lse−1): vertical black bands

The resulting image is then reset to the screen resolution to bedisplayed in full screen mode. For example, with an image of resolution1920×1080, the effective width (Lse) should be 45 cm and is presented ona screen whose final width (Ls1′) is 60 cm. An image may be as follows:Rimg_h′=60 cm/45 cm*1920 pixels=2560 pixels(Rimg_h′−Rimg_h)/2=320 nlack pixels to be added on each sideRimg_v′=60 cm/45 cm*1080 pixels=1440 pixels(Rimg_v′−Rimg_v)/2=180 pixels to add vertically at the top and bottom ofthe image

The final image will therefore have a resolution of 2560×1440 pixelswith a preserved aspect ratio of 1.78:1. This new image can then bereset to the screen resolution to display in full screen mode. Forexample, if the screen had a resolution of 800 pixels, then the activepart (displaying image data) would be 1920/2560*800=600 pixels.

Method 2:

Alternatively, an image can be created that would be presented in awindow centered horizontally in the screen, preferably verticallycentered as well. The image has the following resolution:Rimg_h′=Lse/Ls1′*Rs_hRimg_v′=Rimg_v*Rimg_h′/Rimg_h

Taking the same example as earlier with a 45 cm Lse, a 60 cm Ls1, ahorizontal image resolution of 1920 pixels and a screen of 800 pixelswide. Therefore:Rimg_h′=45 cm/60 cm*800 pixels=600 pixels

The image (downscale) is reduced from 1920 pixels to 600 pixels and tocenter it in the screen which gives exactly the same result as above(active part of the image).

Step 3(B): Image Adjustment when Lse>Ls1′

When the effective width of the image is greater than the effectivewidth of the screen, the image is reduced by cutting also on each sideof the image to maintain the horizontal centering. The following methodcan be used:Rimg_h′=Rimg_h/Lse*Ls1′Rimg_h−Rimg_h′=Rimg_h*(1−Ls1′/Lse)(Rimg_h−Rimg_h′)/2=Rimg_h/2*(1−Ls1′/Lse): number of pixels to cut oneach sideANDRimg_v′=Rimg_v/Lse*Ls1′(Rimg_v−Rimg_v′)/2=Rimg_v/2*(1−Ls1′/Lse): number of pixels to cut oneach side

For example, an image with a horizontal resolution of 1920 pixels thatshould have an effective width of 50 cm (Lse) at the distance from thescreen but whose actual screen width is only 30 cm. The image may be cutas follows:Rimg_h′=1920 pixels/50 cm*30 cm=1152 pixelsNumber of pixels to cut on each side=1920 pixels/2*(1−30 cm/50 cm)=384pixelsRimg_v′=1080 pixels/50 cm*30 cm=648 pixelsNumber of pixels to cut on each side=1080 pixels/2*(1−30 cm/50 cm)=216pixels

The final image would therefore have a resolution of 1152×648 pixelswith the same aspect ratio of 1.78:1. All that remains is to adjust theresolution of the image thus obtained to the resolution of the screen topresent it in full screen mode.

Method Adaptation for Parallel Screens

A single screen system is now studied.

As shown in FIGS. 15A, 15B and 15C, a user is looking at an image on atelevision offering a field of view more limited to the distance fromwhere the user sits to watch the screen. As seen earlier, the finalscreen width is given by the formula Ls1′=Ls+Io.

To adjust the image on the screen, two of the following steps can beperformed:

STEP 1: Adjust the image resolution (Rimg_h and Rimg_v) of the twoimages (left and right eye) so that the images respect the initial ratioDs/Lse

STEP 2: Cut the right-hand portions of the image of the left and righteye of the new eye obtained from the technique of section 5

Let us take the example of an observer with an interocular distance of6.5 cm and looking at a television of 140 cm width (Ls), a 1920×1080pixel resolution image which should have a width of 200 cm (Lse). Step 1will first be carried out.

Step 1:

Ls1′ is first determined, which is equal to Ls+Io, that is 146.5 cm.Since Lse is greater than Ls1′, the images of the left eye and the righteye by are reduced the following method:Rimg_h′=Rimg_h/Lse*Ls1′=1920 pixels/200 cm*146.5 cm=1406 pixels(Rimg_h′−Rimg_h)/2=Rimg_h/2*(1−Ls1′/Lse): number of pixels to cut oneach side=1920 pixels/2*(1−146.5 cm/200 cm)=257 pixelsANDRimg_v′=Rimg_v/Lse*Ls1′=1080 pixels/200 cm*146.5 cm=791 pixels(Rimg_v′−Rimg_v)/2=Rimg_v/2*(1−Ls1′/Lse): number of pixels to cut oneach side=1080 pixels/2*(1−146.5 cm/200 cm)=144.5 pixels

The intermediate image has therefore a resolution of 1406×791 pixelsretaining the same initial aspect ratio of 1.78:1. Step 2 is now carriedout.

Step 2:

The right part of the left eye and the left part of the right eye arecut using the intermediate image as the basis of calculation as follows:Io/(Ls+Io)*Rimg_h′6.5 cm/(140 cm+6.5 cm)*1406 pixels=62.3823 pixels

To maintain the aspect ratio of the original image, the image is cutalong the vertical axis by the same proportion:Io/(Ls+Io)*Rimg_v′6.5 cm/(140 cm+6.5 cm)*791 pixels=35,0956 pixels

The final image for the left eye will therefore be a section of theoriginal image (part of the right) with a resolution of 1344×756 pixelsand with an aspect ratio of 1.78:1. The image of the right eye will becomposed of the equivalent left section of the original image of theright eye, i.e. a section of 1344×756 with the same vertical alignmentas the section of the left eye. All that remains is to adjust theresolution of the left eye and right eye images to that of the screen toobtain the final images to be displayed in full screen mode.

Stereoscopic Zoom: Changing the Image Size (Lse′)

In monoscopy, a zoom corresponds to a magnification of an image in the xand y axes by a given factor. If a zoom of 2 is made, then the imagewill appear twice as large as the original image. On the other hand, asseen previously, such a magnification in stereoscopy will have an effectnot only on the size of the image on the screen but also on theperceived depth of objects.

With an example of a stereoscopic image presented to scale for a givenscreen (Z=1, Io/B=1), the pair of stereoscopic images (left and rightimage) are modified identically by a factor of X so that Lse′/Lse=X. Theimpact of this change for a given user staying at the same distance fromthe screen is observed.

Impact on Perceived Distance

According to the equations established above, it can be establishedthat:

$\frac{{Dp}^{\prime}}{Dp} = {\frac{{{Io}/B}*{{Ds}/{Lse}^{\prime}}*{{Lc}/F}*{Drn}}{{{Io}/B}*{{Ds}/{Lse}}*{{Lc}/F}*{Drn}} = {\frac{Lse}{{Lse}^{\prime}} = \frac{1}{X}}}$

So for an image magnification factor of X, the perceived distance ofobjects will be reduced proportionally by 1/X.

Impact on the Perceived Width

According to the equations established above, it can also be establishedthat:

$\frac{{Lp}^{\prime}}{Lp} = {\frac{{Lse}^{\prime}*{{Dp}^{\prime}/2}{Ds}}{{Lse}*{{Dp}/2}{Ds}} = {{\frac{{Lse}^{\prime}}{Lse}*\frac{{Dp}^{\prime}}{Dp}} = {{X*\frac{{Dp}^{\prime}}{Dp}} = {{X*\frac{1}{X}} = 1}}}}$

So for an image magnification factor of X, the perceived width of theobjects will be unchanged.

Impact on the Proportionality

Finally, according to the equations established above, it can beestablished that:

$\frac{Z^{\prime}}{Z} = {\frac{{{Ds}/{Lse}^{\prime}}*{{Lc}/F}}{{{Ds}/{Lse}}*{{Lc}/F}} = {\frac{Lse}{{Lse}^{\prime}} = \frac{1}{X}}}$

So for an image magnification factor of X, the proportionality scalewill be changed by an inversely proportional factor of 1/X.

In summary (see the graphs of FIGS. 18A, 18B and 18C):

Original Zoom in Zoom out Effective X = 1 X > 1 X < 1 width Lse′ = LseLse′ > Lse Lse′ < Les Scale of Épr = 1 Épr = 1/X < 1 Épr = 1/X > 1 depthObjects perceived Objects perceived Objects at the same distance closerthan in the perceived as in the real world real world farther than inthe real world Spatial scale Ésp = 1 Ésp = 1 Ésp = 1 Objects appear ofthe same width as in the real world Scale of Z = 1 Z = 1/X < 1 Z = 1/X >1 proportion- Proportional or Flattened world Stretched World alityorthostereoscopy

In order to preserve the proportionality of the stereoscopicrepresentation, the change in the perceived distance of the image isaccompanied by an equal and proportional change in the perceived size ofthe image. In other words, the variation of the spatial scale (Esp=Io/B)is equal to the variation of the depth scale (Epr) so that the scale ofproportionality remains equal to 1.

However, the components of the spatial scale (Io and B) cannot bemodified because the base of the stereoscopic camera system has alreadybeen fixed to the shoot and the distance between the two eyes of theuser can obviously not be modified. In other words, there is no way ofpreserving the proportionality of the experience once the scale ormagnification of the image on the screen is changed.

So for a zoom with the image magnification method:

A zoom in will allow «entering» in the 3D world. The field of view willbe smaller and the 3D world will flatten.

A zoom out will allow a user to retreat from the 3D world. The field ofview will be wider and the 3D world will stretch.

Stereoscopic Zoom: Changing the Optical Base (Bo)

The following exemplary illustration is provided. When a zoom is made,there is globally a change in the scale (x, y, z) by a factor X suchthat:

${{Delta}\mspace{14mu} Z} = {X = {\frac{{Dp}’}{Dp} = \frac{{Lp}’}{Lp}}}$

If it is true that one can not change the interocular distance of a user(Io), one can however change the positioning of the images with respectto the center of the optical axis of the two eyes. Optical base (Bo) isdefined as the distance between the center of the two images on thescreen. It can be shown how the basic optical change impacts theperceived width and perceived depth of objects.

Impact on Perceived Width

FIG. 19A illustrates the geometry of Io and Bo for the images displayedon the screen, and FIG. 19B illustrates the resulting change in objectwidth as a result of modifying Bo.

The optical base is positioned so that its middle is perfectly centeredbetween both eyes of the observer.

It can be established that:

${\frac{{La} + {Lp}^{\prime}}{{Dp}^{\prime}} = {\frac{{{Lse}/2} + \left( {{{Io}/2} - {{Bo}/2}} \right)}{Ds} = \frac{{Lse} + \left( {{Io} - {Bo}} \right)}{2{Ds}}}}{AND}{\frac{La}{{Dp}^{\prime}} = {{\frac{\left( {{{Io}/2} - {{Bo}/2}} \right)}{Ds}\mspace{14mu}{OR}\mspace{14mu}{La}} = \frac{{Dp}^{\prime}\left( {{Io} - {Bo}} \right)}{2{Ds}}}}$

By replacing [La] in the first equation with the result of the second,it can be established that:

${\frac{{La} + {Lp}^{\prime}}{{Dp}^{\prime}} = \frac{{{{{Dp}^{\prime}\left( {{Io} - {Bo}} \right)}/2}{Ds}} + {Lp}^{\prime}}{{Dp}^{\prime}}}{\frac{{{{{Dp}^{\prime}\left( {{Io} - {Bo}} \right)}/2}{Ds}} + {Lp}^{\prime}}{{Dp}^{\prime}} = \frac{{Lse} + \left( {{Io} - {Bo}} \right)}{2{Ds}}}{{\frac{\left( {{Io} - {Bo}} \right)}{2{Ds}} + \frac{{Lp}^{\prime}}{{Dp}^{\prime}}} = \frac{{Lse} + \left( {{Io} - {Bo}} \right)}{2{Ds}}}{\frac{{Lp}^{\prime}}{{Dp}^{\prime}} = {{\frac{Lse}{2{Ds}} + \frac{\left( {{Io} - {Bo}} \right)}{2{Ds}} - \frac{\left( {{Io} - {Bo}} \right)}{2{Ds}}} = \frac{Lse}{2{Ds}}}}{SO}{{Lp}^{\prime} = \frac{{Lse}*{Dp}^{\prime}}{2{Ds}}}$

The proportion ratio Lp′ on Lp can now be established:

$\frac{{Lp}^{\prime}}{Lp} = {\frac{{Lse}*{{Dp}^{\prime}/2}{Ds}}{{Lse}*{{Dp}/2}{Ds}} = \frac{{Dp}^{\prime}}{Dp}}$

So for a given image width, the perceived change in the width of anobject will be equal to the perceived change in the distance of thatobject. The change of the optical base allows the condition ofproportionality of the scale to be met. How a change in the optical baseaffects the perceived distance of objects in the stereoscopicrepresentation will now be described.

Impact on the Perceived Distance

An object located respectively at the points Ag and Ad of the image ofthe left and right eye is taken. FIGS. 20A and 20B show the impact whenusing an optical base Bo lower than Io.

Based on the properties of the rectangle triangles, it can beestablished that:

$\left. {{\frac{A + B}{{Dp}^{\prime}} = {\frac{C + D}{{Dp} - {Ds}}\mspace{14mu}{YET}\mspace{14mu}\begin{matrix}{{A + B} = {Io}} \\{{C + D} = {{Bo} - {Ps}}}\end{matrix}}}{SO}{\frac{Io}{{Dp}^{\prime}} = {{\frac{{Bo} - {Ps}}{{Dp}^{\prime} - {Ds}}\mspace{14mu}{OR}\mspace{14mu}\frac{{Dp}^{\prime}}{Io}} = \frac{{Dp}^{\prime} - {Ds}}{{Bo} - {Ps}}}}{{{Dp}^{\prime}*\left( {{Bo} - {Ps}} \right)} = {{{Io}*{Dp}^{\prime}} - {{Io}*{Ds}}}}} \right){{{{Dp}^{\prime}\left( {{Bo} - {Ps}} \right)} - {{Dp}^{\prime}*{- {Io}}}} = {{- {Ds}}*{Io}}}{{{Dp}^{\prime}\left( {{Bo} - {Ps} - {Io}} \right)} = {{- {Ds}}*{Io}}}{{Dp}^{\prime} = {\frac{{- {Ds}}*{Io}}{{Bo} - {Ps} - {Io}} = \frac{{Ds}*{Io}}{{Ps} + {Io} - {Bo}}}}$

The proportion ratio Dp′/Dp can be established as follows:

$\begin{matrix}{{Dp}^{\prime} = {{\frac{{Ds}*{Io}}{{Ps} + {Io} - {Bo}}\mspace{14mu}{AND}\mspace{14mu}{Dp}} = \frac{{Ds}*{Io}}{Ps}}} \\{\frac{{Dp}^{\prime}}{Dp} = {{{Dp}^{\prime}*\frac{1}{Dp}} = {\frac{{Ds}*{Io}}{{Ps} + {Io} - {Bo}}*\frac{Ps}{{Ds}*{Io}}}}} \\{\frac{{Dp}^{\prime}}{Dp} = \frac{Ps}{{Ps} + {Io} - {Bo}}}\end{matrix}$

This relationship demonstrates that when using a different optical base,the orthostereoscopic effect is lost. Indeed, while variations of Dp arelinear, the variations of the ratio Dp′/Dp are not linear, since theyvary according to Ps, namely, they vary as a function of the distance ofthe objects which were captured in the real world. For a unit variationof Drn, the variation of Dp′ will change according to the value of Drn.This can be seen as a number of zones in which there is approximatelylinearly proportional variation. With this relationship, the value of X(3D magnification ratio) can now be calculated:

$\begin{matrix}{\frac{{Dp}^{\prime}}{Dp} = {\frac{Ps}{{Ps} + {Io} - {Bo}} = X}} & {{{YET}\mspace{14mu}{Lse}} = {{Ds}*{{Lc}/F}}} \\{\frac{1}{X} = \frac{{Ps} + {Io} - {Bo}}{Ps}} & {\frac{1}{X} = {1 + \frac{{Io} - {Bo}}{\left( {B*F*{Ds}*{{Lc}/F}} \right)/\left( {{Drn}*{Lc}} \right)}}} \\{\frac{1}{X} = {1 + \frac{\left( {{Io} - {Bo}} \right)}{Ps}}} & {{\frac{1}{X}1} + \frac{{Io} - {Bo}}{\left( {B*{{Ds}/{Drn}}} \right)}} \\{\frac{1}{X} = {1 + \frac{{Io} - {Bo}}{\left( {{{B.F.{Lse}}/{Drn}}*{Lc}} \right)}}} & {X = \frac{1}{1 + \frac{\left( {{Io} - {Bo}} \right)}{\left( {B*{{Ds}/{Drn}}} \right)}}}\end{matrix}$

This result demonstrates that the magnification factor X is only validfor a specific real distance Drn. For example, if a 3D magnificationequivalent to one third of the original representation is obtained, anoriginal distance (Drn) in order to achieve this relationship may bespecified. Arbitrarily, the distance Drn is chosen as a real referencedistance which, in orthostereoscopic mode, is displayed in the zeroplane that is to say at the distance from the screen (Dp=Ds). The resultis:

$\begin{matrix}\; & \begin{matrix}\begin{matrix}{{Dp} = {{Ds} = {\begin{matrix}{Io} \\B\end{matrix}*\begin{matrix}{Ds} \\{Lse}\end{matrix}*\begin{matrix}{Lc} \\F\end{matrix}*{Drn}}}} \\{{{In}\mspace{14mu}{ortho}},{{{Ds}/{Lse}} = {F/{Lc}}}} \\{{Ds} = {\begin{matrix}{Io} \\B\end{matrix}*\begin{matrix}F \\{Lc}\end{matrix}*\begin{matrix}{Lc} \\F\end{matrix}*{Drn}}} \\{{Ds} = {\begin{matrix}{Io} \\B\end{matrix}*{Drn}}} \\{{Drn} = {{Ds}*\frac{B}{Io}}}\end{matrix} & \begin{matrix}{SO} \\{{B*\frac{Ds}{Drn}} = {{B*\frac{Ds}{{Ds}*{B/{Io}}}} = {Io}}} \\{AND} \\{X = \frac{1}{1 + {\left( {{Io} - {Bo}} \right)/{Io}}}} \\{X = \frac{1}{1 + 1 - {{Bo}/{Io}}}} \\{X = \frac{1}{2 - {{Bo}/{Io}}}}\end{matrix}\end{matrix}\end{matrix}$

Conversely, when Bo is to be determined, giving a desired 3Dmagnification, Bo may be isolated as follows:

$\begin{matrix}{X = \frac{1}{2 - {{Bo}/{Io}}}} \\{\frac{1}{X} = {2 - \frac{Bo}{Io}}} \\{\frac{Bo}{Io} = {2 - \begin{matrix}1 \\X\end{matrix}}} \\{{Bo} = {{Io}*\left( {2 - \frac{1}{X}} \right)}}\end{matrix}$

The graphs of FIGS. 21A and 21B illustrate the impact of a modificationof the optical base on the ratio between the distances of the real worldand the perceived world.

As shown in the graphs above, the perceived space is not linear (ratioDp′/Drn not constant) in addition to not being orthostereoscopic. Whenzooming inside the image by changing the optical base, a plateau israpidly reached in the perceived distances. The maximum distanceperceived (when Drn=infinity) is calculated as follows:

$\begin{matrix}{{Ps} = \frac{B*F*{Lse}}{{Drn}*{Lc}}} \\{{{{WHEN}\mspace{14mu}{Drn}} = {infinity}},{{Ps} = {0\mspace{14mu}{So}}}} \\{{{Dp}^{\prime}{MAX}} = {\frac{{Ds}*{Io}}{{Ps} + {Io} - {Bo}} = \frac{{Ds}*{Io}}{{Io} - {Bo}}}}\end{matrix}$

Method Limitations

First of all, it is not possible to zoom out because there would bedivergence in objects at distances from Drn to infinity. An object atinfinity would normally be represented in the center of each eye whenBo=Io. If Bo is greater than Io, then the points will be found to theleft of the left eye and to the right of the right eye, respectively.Since the eyes cannot diverge, this method would make fusion impossibleand cause pain for the user.

Also, this method does not significantly increase the portion of theperceived image. Since the images are only moved a few centimeters, theexpected effect cannot be achieved by zooming (i.e. significantly changethe field of view).

Finally, the optical base modification leads to significant spatialdistortions and causes the loss of space linearity and orthostereoscopiceffect. For all these reasons, changing the optical base is not therecommended method for 3D zooming of a stereoscopic image.

Comfortable Stereoscopic Representation Taking into Account theManagement of Vergence and Accommodation

To establish the depth of an object, the brain uses many visual cuesthat it combines together to obtain a greater level of certainty. Forexample, interposition, motion parallax, blur, perspective lines and, ofcourse, stereopsis (parallax) can be used. In conventional/monoscopicvideo games and cinema, many of these techniques are used to give agreater sense of depth to the content and sometimes to provide animpression of depth to the spectators/players.

In the case of stereoscopic content, an impression of depth is given byusing a parallax difference. However, parallax information oftenconflicts with other visual cues. For example, one case is that of anobject that should be in front of a screen but whose image is “cut off”by the edge of the screen as illustrated in FIG. 22.

In the image of FIG. 22, the baseball should come out of the screenaccording to the stereoscopic information, however, it “touches” theframe, that is to say that the frame of the screen seems to block theimage of the ball. But in everyday life, the brain has learned that anobject that visually blocks another object is in front of it (phenomenonof interposition). So there is a conflict between visual cues, and sinceinterposition is commonly used by the visual cortex, the brain decidesto reject stereoscopic information and position the ball at the screen(e.g. will refuse to perceive it in front of the screen). Stereographersare familiar with this phenomenon and are careful to correctly frame theobjects that are to appear in front of the screen.

The other principal issue comes from the difference between vergenceinformation (where the eyes converge) and accommodation (distance towhich the eyes focus). The brain regularly manages these two pieces ofinformation concurrently to allow for clear vision. These two pieces ofinformation are supposed to be in agreement with each other and thebrain uses both pieces of information together to make better decisions(adjustments). In stereoscopy, these two pieces of information are notalways in agreement because although convergence is achieved at a givendistance (Dp), the eyes will focus at the distance of the screen (Ds).FIG. 23A illustrates schematically the vergence distance and the focaldistance that are the same for real world viewing, and FIG. 23Billustrates schematically the vergence distance and the closer focaldistance of the screen in stereoscopic 3D viewing.

It has been shown in the literature that when there is too much conflictbetween vergence and accommodation in stereoscopy, many adverse effectsmay occur such as discomfort, pain (sometimes persistent) and diplopia(double-vision, no fusion). This conflict between vergence andaccommodation has not only an impact on the level of comfort but also onthe perception of depth of objects in a stereoscopic representation.

Experimentation has been conducted with parallel cameras as well as withcomputer-generated objects placed at various distances. At the time ofthe observation, it has been observed that despite the importantdifferences of parallax (measured and validated on the screen), theperception of the distance of the object changed only modestly for thepositioning of objects very far from the screen. When there is aconflict between the vergence and accommodation information, the humanbrain may give precedence to the accommodation information and theperceived distance of objects will be related to the distance from thescreen. This effect may be accentuated if there are many objects in thefield of view near the screen corroborating the accommodationinformation.

In order to manage this problem, the maximum or farthest distance(perceived «inside» the screen) and minimum distance («out» of thescreen) respecting the angular constraint are determined.

Maximum Distance (Df)

According to the article “Visual Discomfort and Visual Fatigue ofStereoscopic Displays: A Review” by Marc Lambooij et al., published inthe Journal of Imaging Science and Technology, dated May-June 2009(53(3): 030201-030201-14, 2009), it is proposed to respect a limit of 1°between the angle formed by the eyes when they converge on the screen(“accommodation” angle) and the maximum or minimum convergence angle tomaintain a comfortable experience. This principle is used as a basis fordetermining the maximum and minimum distance of stereoscopic perception.It is important to note that there is an important difference betweenvisualization with lenses (e.g., stereoscopes, virtual realityheadsets), where the accommodation is done at infinity, and conventionalscreens where the accommodation is done at the distance of the screen.The case of conventional screens will first be described.

FIG. 24A represents a case of convergence on the inside of the screen.From this figure, finding the value of the distance of an object (Do) isestablished as follows:

$\begin{matrix}{\frac{Do}{Io} = \frac{{Do} - {Ds}}{P}} \\{\frac{P*{Do}}{Io} = {{Do} - {Ds}}} \\{{\frac{P*{Do}}{Io} - {Do}} = {- {Ds}}} \\{{{Do}\left( {\frac{P}{Io} - 1} \right)} = {- {Ds}}} \\{{Do} = \frac{Ds}{\left( {1 - {P/{Io}}} \right)}}\end{matrix}$

With respect to FIG. 24B, when the eyes converge on the screen, theconvergence angle of the left eye is equal to θ. When the eyes converge«inside» the screen, the angle formed is reduced to θ′ for each eye. Thevalue of P that will meet the angular constraint of vergence (V,expressed in radians, is the angle multiplied by π and divided by 180°)while maintaining stereoscopic perception and comfort of viewing is nowdetermined:

$\begin{matrix}{{\tan\;\theta} = {\frac{{Io}/2}{Ds} = \frac{Io}{2{Ds}}}} \\{\theta^{\prime} = {\theta - {{V/2}I\mspace{14mu}\left( {{both}\mspace{14mu}{eye}\mspace{14mu}{converge}} \right)}}} \\{{\tan\;\theta^{\prime}} = {{\tan\left( {\theta - \frac{V}{2}} \right)} = {\frac{{{Io}/2} - {P/2}}{Ds} = \frac{{Io} - P}{2{Ds}}}}} \\{\theta = {{atan}\left( {{{Io}/2}{Ds}} \right)}} \\{{\tan\left( {\theta - {V/2}} \right)} = {\tan\left( {{{atan}\left( {{{Io}/2}{Ds}} \right)} - {V/2}} \right)}} \\{{\tan\left( {{{atan}\left( {{{Io}/2}{Ds}} \right)} - {V/2}} \right)} = \frac{{Io} - P}{2{Ds}}} \\{{2{Ds}*{\tan\left( {{{atan}\left( {{{Io}/2}{Ds}} \right)} - {V/2}} \right)}} = {{Io} - P}} \\{P = {{Io} - {2{Ds}*{\tan\left( {{{atan}\left( {{{Io}/2}{Ds}} \right)} - {V/2}} \right)}}}}\end{matrix}$

Now that the value of P is obtained, fulfilling the condition ofvergence, P can be integrated into the previous equation and the maximumdistance is obtained as follows:

$\begin{matrix}{{Df} = \frac{Ds}{\left( {1 - {P/{Io}}} \right)}} \\{{Df} = \frac{Ds}{1 - \frac{\left( {{Io} - {2{Ds}*{\tan\left( {{{atan}\left( {{{Io}/2}{Ds}} \right)} - {V/2}} \right)}}} \right.}{Io}}} \\{{Df} = \frac{{Ds}*{Io}}{2{Ds}*{\tan\left( {{{atan}\left( {{{Io}/2}{Ds}} \right)} - {V/2}} \right)}}} \\{{Df} = \frac{Io}{2*{\tan\left( {{{atan}\left( {{{Io}/2}{Ds}} \right)} - {V/2}} \right)}}}\end{matrix}$

Starting from a distance to the screen, all objects to infinity can becomfortably merged as they will all be within the vergence constraint.To establish this distance, the value of Ds is established when Df tendsto infinity as follows:

$\begin{matrix}{{Df} = \frac{Io}{2*{\tan\left( {{{atan}\left( {{{Io}/2}{Ds}} \right)} - {V/2}} \right)}}} \\{{\tan\left( {{{atan}\left( {{{Io}/2}{Ds}} \right)} - {V/2}} \right)} = {{{Io}/2}*{Df}}} \\{{{When}\mspace{14mu}{Df}\mspace{14mu}{tends}\mspace{14mu}{to}\mspace{14mu}{infinity}},{{{Io}/{Df}} = {0\mspace{14mu}{{So}:}}}} \\{{\tan\left( {{{atan}\left( {{{Io}/2}{Ds}} \right)} - {V/2}} \right)} = 0} \\{\left. {{{atan}\left( {{{Io}/2}{Ds}} \right)} - {V/2}} \right) = 0} \\{{{{{Io}/2}{Ds}} - {V/2}} = 0} \\{{{{Io}/2}{Ds}} = {V/2}} \\{{Ds} = {{Io}/V}}\end{matrix}$

Taking an example of a person with an interocular distance of 6.5 cm andwhere a limit of vergence of 2° or 2°π/180° in radians is respected,then the distance to the screen that will allow a comfortable fusion toinfinity would be:Ds=Io/VDs=6.5 cm/(2°π/180°)Ds=186.21 cm

This demonstrates that for stereoscopic representations on screens at arelatively close distance from an average user, the stereoscopic effectcan have a natural and very important depth (up to infinity). Thiscorresponds well to the projections in room as well as to the viewing ontelevisions 3D. On the other hand, for viewing on stereoscopic screenscloser to the user (eg, mobile phones, computer screens, tablets, etc.),there are serious depth limitations. For example, for the same user asin the previous example, a screen is placed at 60 cm from the user (e.g.a laptop), the maximum acceptable depth under the constraint of 2° wouldbe 88.6 cm or only 28.6 cm inside the screen which is very limiting.

Minimum Distance (Dn)

With reference to FIGS. 25A and 25B, the minimum distance perceived by auser of an object coming out of the screen is now calculated, that is tosay positioned between the user and the screen.

FIG. 25A represents a case of convergence in front of the screen. Fromthis figure, finding the value of the distance of an object (Do) can beestablished as follows:

$\begin{matrix}{\frac{Do}{Io} = \frac{{Ds} - {Do}}{P}} \\{\frac{P*{Do}}{Io} = {{Ds} - {Do}}} \\{{\frac{P*{Do}}{Io} + {Do}} = {Ds}} \\{{{Do}\left( {\frac{P}{Io} + 1} \right)} = {Ds}} \\{{Do} = \frac{Ds}{\left( {{P/{Io}} + 1} \right)}}\end{matrix}$

With reference to FIG. 25B, when the eyes converge on the screen, theconvergence angle of the left eye is equal to θ. When the eyes convergein front of the screen, the angle formed is increased to θ′ for eacheye. The value of P that will respect the vergence angular constraint(V, expressed in radians, is the angle multiplied by π and divided by180°) while maintaining a stereoscopic perception and comfort of viewingis then determined:

$\begin{matrix}{{\tan(\theta)} = {\frac{{Io}/2}{Ds} = \frac{Io}{2{Ds}}}} \\{\theta^{\prime} = {\theta - {{V/2}\mspace{14mu}\left( {{{both}\mspace{14mu}{eyes}\mspace{14mu}{converge}},{{angles}\mspace{14mu}{in}\mspace{14mu}{radians}}} \right)}}} \\{{\tan\;\theta^{\prime}} = {{\tan\left( {\theta + \frac{V}{2}} \right)} = {\frac{{{Io}/2} + {P/2}}{Ds} = \frac{{Io} + P}{2{Ds}}}}} \\{\theta = {{atan}\left( {{{Io}/2}{Ds}} \right)}} \\{{\tan\left( {\theta - {V/2}} \right)} = {\tan\left( {{{atan}\left( {{{Io}/2}{Ds}} \right)} + {V/2}} \right)}} \\{{\tan\left( {{{atan}\left( {{{Io}/2}{Ds}} \right)} + {V/2}} \right)} = \frac{{Io} + P}{2{Ds}}} \\{{2{Ds}*{\tan\left( {{{atan}\left( {{{Io}/2}{Ds}} \right)} + {V/2}} \right)}} = {{Io} + P}} \\{P = {{2{Ds}*{\tan\left( {{{atan}\left( {{{Io}/2}{Ds}} \right)} + {V/2}} \right)}} - {Io}}}\end{matrix}$

Now that the value of P corresponding to the constraint of vergence isdetermined, the P value can be integrated into the preceding equationand the minimum distance (Dn) is obtained as follows:

$\begin{matrix}{{Dn} = \frac{Ds}{\left( {{P/{Io}} + 1} \right)}} \\{{Dn} = \frac{Ds}{\frac{\left( {{2{Ds}*{\tan\left( {{{atan}\left( {{{Io}/2}{Ds}} \right)} + {V/2}} \right)}} - {Io}} \right)}{Io} + 1}} \\{{Dn} = \frac{{Ds}*{Io}}{2{Ds}*{\tan\left( {{{atan}\left( {{{Io}/2}{Ds}} \right)} + {V/2}} \right)}}} \\{{Dn} = \frac{Io}{2*{\tan\left( {{atan}\left( {{{{Io}/2}{Ds}} + {V/2}} \right)} \right.}}}\end{matrix}$

Parameterization to Respect the Vergence-Accommodation Conflict

The maximum and minimum distances of a stereoscopic representationrespecting the vergence-accommodation conflict have been determined tobe:

$\begin{matrix}{{Far}\mspace{14mu}{{Distance}:}} \\{{Df} = \frac{Io}{2*{\tan\left( {{{atan}\left( {{{Io}/2}{Ds}} \right)} - {V/2}} \right)}}} \\{{Close}\mspace{14mu}{{Distance}:}} \\{{Dn} = \frac{Io}{2*{\tan\left( {{{atan}\left( {{{Io}/2}{Ds}} \right)} + {V/2}} \right)}}}\end{matrix}$

It has been shown that a modification of the field of view (Lse′) cannotreduce the total depth of the perceived world in the stereoscopicrepresentation. Indeed, a point at infinity captured with parallelcameras will be perceived to infinity in the stereoscopic representationregardless of how the stereoscopic field of view is changed (alwayscentered on each eye). On the other hand, it has been demonstrated thatthe depth of the perceived world can be reduced in the representation byaltering the optical base of the system.

The optical base in order to respect the Df constraint is nowdetermined. The most distant point captured by the parallel camerasystem (Drn=infinity) is perceived in stereoscopic representation at themaximum distance allowing a comfortable experience (Dp′=Df):

$\begin{matrix}{{{{When}\mspace{14mu}{Drn}} = {infinity}},{{Ps} = {0\mspace{14mu}{So}}}} \\{{Df} = {{Dp}^{\prime} = {\frac{{Ds}*{Io}}{{Ps} + {Io} - {Bo}} = \frac{{Ds}*{Io}}{{Io} - {Bo}}}}} \\{{{Io} - {Bo}} = \frac{{Ds}*{Io}}{Df}} \\{{Bo} = {{Io} - \frac{{Ds}*{Io}}{Df}}} \\{{Bo} = {{Io} - {{Ds}*\frac{Io}{\frac{Io}{2*{\tan\left( {{{atan}\left( {{{Io}/2}{Ds}} \right)} - {V/2}} \right)}}}}}} \\{{Bo} = {{Io} - {{Ds}*2*{\tan\left( {{{atan}\left( {{{Io}/2}{Ds}} \right)} - {V/2}} \right)}}}}\end{matrix}$

Note that this adjustment is made for any distance to the screen lessthan the minimum distance allowing a fusion at infinity so that Ds<Io/V(V expressed in radians). For any screen distance greater than Io/V, theoptical base can be set to Io.

When the optical base is established at a value less than Io, thelinearity of space is also changed. One consequence of this change inspace is that objects in orthostereoscopy would normally end up at thedistance of the screen are now «out» of the screen. Thus, a portion ofthe image that should be inside the screen now comes out of the screenwhich causes discomfort and creates framing problems.

In order to solve this problem, a modification of the image size (Lse)can be used so that the real distance of the objects presented in thezero plane (distance from the screen) is the equivalent to the scaleorthostereoscopic representations. For example, in the case of an imagecaptured at a scale of 1 (proportional to the natural world) whosestereoscopic representation would be on a screen located at 60 cm from auser, it is preferable that an object perceived at a distance of 60 cmfrom the screen be located at 60 cm from the camera when the image withthe object was captured.

To do this, the real distance of an object presented on the screen isestablished in the case of an orthostereoscopic representation withBo=Io. This distance can be calculated as follows: Drn=Ds*B/Io. Theimage width Lse′ that will allow the perceived distance (Dp′) to beequal to the distance to the screen (Ds) and Drn is established. It canbe determined as follows:

$\begin{matrix}\begin{matrix}{{Dp}^{\prime} = {{Ds} = \frac{{Ds}*{Io}}{{Ps} + {Io} - {Bo}}}} \\{1 = \frac{Io}{{Ps} + {Io} - {Bo}}} \\{{{Ps} + {Io} - {Bo}} = {Io}} \\{{Ps} = {{Io} - {Io} + {Bo}}} \\{{Ps} = {Bo}}\end{matrix} & \begin{matrix}{{Ps} = \frac{B*F*{Lse}^{\prime}}{{Drn}*{Lc}}} \\{{Bo} = \frac{B*F*{Lse}^{\prime}}{{Drn}*{Lc}}} \\{{{YET}:{Drn}} = {{Ds}*{B/{Io}}}} \\{So} \\{{BO} = \frac{B*F*{Lse}^{\prime}}{{Ds}*{B/{Io}}*{Lc}}} \\{\frac{{Ds}*B}{Io} = \frac{B*F*{Lse}^{\prime}}{{Lc}*{Bo}}} \\{{Lse}^{\prime} = \frac{{Ds}*{Bo}*{Lc}}{{Io}*F}}\end{matrix}\end{matrix}$

Note that when Bo is equal to Io (when the user is at a sufficientlylarge distance from the screen), Lse′ becomes equal to Lse, resulting ina return to orthostereoscopic mode.

FIGS. 16A and 16B are schematic block diagrams of a device forprocessing parallel camera stereoscopic video to adapt to differentviewing conditions. The capture parameters allow the determining of theoriginal field of view. These parameters can be encoded in the images orvideo stream, set by a user or detected by video analysis. Block 12 thusrepresents a memory store of the capture field of view parameters andoptionally includes an interface to receive field of view parametersfrom the image data store or the video streams 22 a and 22 b.

The display/screen parameters can be the screen distance, screenresolution, screen size and interocular distance of the viewer. Theseparameters can be stored in a memory 14. While the interocular distancecan be a variable set in memory 14, it can also be fixed at a nominalvalue within calculator 20 that determines crop and scale parameters asdescribed in detail above. When a screen is shared by multiple viewers,the interocular distance can be chosen to be that of the person havingthe smallest interocular distance to avoid divergence problems for thatperson.

Calculator 20 can also take into consideration the vergence constraintas described above with reference to FIGS. 17 to 25 to determine cropand scale parameters that will modify the base offset to bring distantobjects closer to the screen and to scale the images with a view toreduce the vergence angle difference between the screen and the objectsseen.

The distance between the viewer and the screen can be input using a userinterface or other suitable way.

In the case that there is a change in the interocular distance, thescale parameters include an image shift parameter, even if other viewconditions respect the original recording. However, if a 3D scene isviewed on a display smaller/larger than an original field of view, thescale parameters include an image shift to maintain the base distancebetween the center of each image on the different size display.

The 3D images, namely the right eye and left eye images stored in stores22 a and 22 b, are accordingly shifted, scaled and cropped/border paddedas required in an image processor 25 as for example is schematicallyillustrated in FIG. 16B. The image processor can be a GPU, CPU, FPGA orany other suitable processing device. The source of the images 22 a and22 b can be a stereographic image stream as is known in the art.

As described above, the stereoscopic viewing can be done using knowntechniques. In the block diagram of FIG. 16B, stereoscopic formatting isdone in block 28. Such image processing can be done in a CPU, however,it can also be performed for example using a GPU or an FPGA. Inanaglyphic presentation, color filter glasses are worn by the viewer andthe screen image is composed of both right and left image data that iscolor encoded. In page-flip operation, the viewer can wear shutterglasses that allow the right and left eyes to see in alternating timeslots while the screen image alternates between right and left images.In an autostereoscopic display, the viewer does not need any glasses,but the screen includes a lens or screen mask that allows the right eyeto see right eye pixels and the left eye to see left eye pixels. In apolarized line-interleave display, odd and even lines have differentpolarization of light (the pattern of pixels of each polarization neednot be limited to alternating horizontal lines), and polarizationglasses are worn so that one eye sees odd lines while the other seeseven lines. The stereoscopic formatting for the desired displaytechnique is done, as shown schematically in FIG. 16B, by a formattermodule 28 prior to transmitting a display signal to the display device.The stereoscopic formatter operations or functions can be done withinthe image processors, if desired. The formatted image or images are thendisplayed using a corresponding display device 30.

In the embodiment of FIG. 16C, the image source is a large field of viewsource, such as a wide angle (e.g. 180- to 360-degree panoramic source),a fish eye lens or a computer-generated image source, that is able toprovide a desired image for a given viewing direction by cropping orde-warping and cropping. The viewing direction module 18 can be part ofa user interface to allow a user to select the viewing direction. Thecropping or de-warping and cropping process is known in the art and isdone in module 19. As illustrated, a source fish-eye camera image is notpresentable as a 2D image until it is de-warped. The de-warping module19 can alternatively be integrated into the image processor 25 so thatthe cropping and scaling required involves selecting the portion of thesource image to be de-warped.

It will be appreciated that the image processing, namely cropping andscaling, can be performed using a volume reader. By volume reader, it ismeant to place the original images in 3D space so as to respect theoriginal capture parameters and to capture virtual “camera” views of theoriginal images with the correct positioning of the points of view orcameras. This can be done within most conventional GPU's for example.

Details are as follows.

Nomenclature

-   D=distance-   L=width-   H=height-   F=focal-   RES=resolution

Original representation Camera (capture) (e.g. stereoscopic) VR ReaderScreen Dc Di Do Ds Lchamps Li Lo Ls Hchamps Hi Ho RESs Fc i = image Lv s= screen Lccd Hv Hccd Fcv c = camera Lcv o = object v = viewport cv =virtual camera

1) Place the Image in the Space (see FIG. 17A)

-   -   Position the left image with an arbitrary width Lo    -   Position the right image with the same width Lo

2) Placing the Camera

-   -   Center the camera on the image, where the x, y, z coordinates        are set to 0.0.0 (on the origin)    -   Place at a distance from the image in order to respect the        ratio:        Do/Lo=Di/Li=Fc/Lccd=Dc/Lchamps        or        Do=(Di*Lo)/Li=(Fc*Lo)/Lccd=(Dc*Lo)/Lchamps

3) Render Images

-   -   Create images of the left eye and right eye using:        -   i) Ratio Lv/Hv=(Ls+Io)/Hs        -   ii) Ratio Fcv/Lccdv=Ds/(Ls+Io)        -   iii) Resolution=RESs*(Ls+Io)/Ls

4) Formatting for Single Screen (see FIG. 17B)

-   -   Overlay the two images    -   Shift the left image to the left by a distance equal to        (Io/2)/(Ls+Io)*Lv    -   Shift the right image to the right by the same distance    -   Keep the common part of the two images    -   Format in anaglyph

Alternatively, in step 4, the image of the left eye (from the left) canbe cut by a number of pixels equal to RESs—the resolution and the righteye image can be cut by the same number of pixels but from the right.

You can zoom in and out by moving the camera closer or farther away instep 2.

In the context of a streaming or online service, the images can beprocessed at a server and transmitted to a remote client display, asillustrated in FIG. 26. In this context, the user at the display or thedisplay equipment can relay the display/screen parameters 14 to theserver where the images are processed and then encoded 41 fortransmission to the client display. A reduction in the data to betransmitted can also be achieved when the crop and scale is performedprior to transmission.

What is claimed is:
 1. A system for streaming stereoscopic imagescomprising: a server comprising a processor and a non-transitory memorystoring instructions, wherein said instructions, when executed by saidprocessor, are operable to perform a method of processing stereoscopicimages for display to a viewer on a single screen of a remote clientdisplay, said stereoscopic images taken using parallel-axis camerashaving a first field of view, the method comprising: receiving and usinga definition of a second field of view provided by said single screen ofsaid remote client display, an interocular distance Io for said viewerand a distance between said viewer and said single screen to positionand to scale said stereoscopic images so that display of said images onsaid single screen of said remote client display at said distance fromsaid viewer respects said first field of view; processing saidstereoscopic images such that when said stereoscopic images as scaledfor said single screen are larger than said single screen, to crop saidimages for said single screen, and when said stereoscopic images asscaled for said single screen are smaller than said single screen,providing a border for said images for said single screen; and encodingsaid processed stereoscopic images for streaming transmission to saidremote client display.
 2. The system as defined in claim 1, wherein saidinstructions further comprise selecting a zoom window within saidstereoscopic images to thus change said first field of view, whereinsaid stereoscopic images are scaled respecting said changed first fieldof view.
 3. The system as defined in claim 2, wherein said zoom windowis offset from a center of said stereoscopic images to permit viewing aregion of interest within said stereoscopic images.
 4. The system asdefined in claim 3, wherein viewer input is used to move said offsetwhile viewing said stereoscopic images.
 5. The system as defined inclaim 1, wherein said stereoscopic images are still images.
 6. Thesystem as defined in claim 1, wherein said stereoscopic images are videoimages.
 7. The system as defined in claim 1, wherein said stereoscopicimages are converted to combined anaglyphic format images.
 8. The systemas defined in claim 1, wherein said stereoscopic images are converted tocolumn interleaved format images for display on an autostereoscopicdisplay.
 9. The system as defined in claim 1, wherein said stereoscopicimages are converted to a sequence of page-flip images for viewing withshutter glasses.
 10. The system as defined in claim 1, wherein saidstereoscopic images are converted to a sequence of line-interleaved forpolarized displays.
 11. The system as defined in claim 1, wherein saidinstructions further comprise acquiring user input to obtain saiddefinition of a second field of view provided by said single screen ofsaid remote client display.
 12. The system as defined in claim 1,wherein said instructions further comprises acquiring sensor data fromsaid remote client display to obtain said definition of a second fieldof view provided by said single screen.
 13. The system as defined inclaim 1, wherein said stereoscopic images are positioned on said singlescreen of said remote client display to correspond to an objectseparation of Io between right eye and left eye images for distantobjects.
 14. The system as defined in claim 13, wherein said viewercomprises a plurality of viewers, and said interocular distance Io isselected to be a smallest interocular distance among said plurality ofviewers.
 15. The system as defined in claim 1, wherein said stereoscopicimages are further scaled and positioned using a relative base offset tocause objects appearing at a maximum depth to appear closer and to causeobjects appearing in front of said single screen to appear closer tosaid single screen so as to restrict at least one of: an interocularangle between focussing at a depth of said single screen and focussingon objects appearing at a modified maximum depth; and an interocularangle between focussing at a depth of said single screen and focussingon objects appearing closest in front of said single screen; so as toreduce eye strain.
 16. The system as defined in claim 15, wherein saidstereoscopic images are further scaled and positioned to restrict bothof an interocular angle between focussing at a depth of said singlescreen and focussing on objects appearing at a modified maximum depthand an interocular angle between focussing at a depth of said singlescreen and focussing on objects appearing closest in front of saidsingle screen.
 17. The system as defined in claim 16, wherein saidstereoscopic images are further scaled to maintain objects appearing ata depth of said single screen to appear at a same depth.
 18. The systemas defined in claim 15, wherein an interocular angle between viewing anobject appearing at a depth on said single screen and objects appearingbehind and/or in front of said single screen is less than approximatelyone degree.
 19. The system as defined in claim 1, wherein saidstereoscopic images comprise panoramic images, said instructions furthercomprising defining a viewing direction within said panoramic images andextracting a portion of said panoramic images using said viewingdirection.
 20. The system as defined in claim 19, wherein said panoramicimages are wide-angle camera lens images, said instructions furthercomprising de-warping at least a portion of said panoramic images. 21.The system as defined in claim 20, wherein said panoramic images arefish-eye lens images.
 22. A computer program product for streamingstereoscopic images comprising: a non-transitory memory storinginstructions for a processor or reconfigurable hardware, wherein saidinstructions, when executed by said processor or reconfigurablehardware, are operable to perform a method of processing stereoscopicimages for display to a viewer on a single screen of a remote clientdisplay, said stereoscopic images taken using parallel-axis camerashaving a first field of view, the method comprising: receiving and usinga definition of a second field of view provided by said single screen ofsaid remote client display, an interocular distance Io for said viewerand a distance between said viewer and said single screen to positionand to scale said stereoscopic images so that display of said images onsaid single screen of said remote client display at said distance fromsaid viewer respects said first field of view; processing saidstereoscopic images such that when said stereoscopic images as scaledfor said single screen are larger than said single screen, to crop saidimages for said single screen, and when said stereoscopic images asscaled for said single screen are smaller than said single screen,providing a border for said images for said single screen; and encodingsaid processed stereoscopic images for streaming transmission to saidremote client display.